Map CwmForCausalLM to llama and LlamaForCausalLM#25611
Map CwmForCausalLM to llama and LlamaForCausalLM#25611ywang96 merged 5 commits intovllm-project:mainfrom
Conversation
Code World Model (CWM) in this collection https://huggingface.co/collections/facebook/cwm-68acbc3eb02570bd89b3aae8 need to be mapped before the Transformers implementation is complete, else we'll have working VLLM mappings that break in HuggingFace. Signed-off-by: Jacob Kahn <jacobkahn1@gmail.com>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run You ask your reviewers to trigger select CI tests on top of Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. 🚀 |
There was a problem hiding this comment.
Code Review
This pull request adds a mapping for the CwmForCausalLM architecture to vLLM's Llama implementation, which is a good step towards supporting Code World Models. The change is straightforward. However, a crucial update to the test registry is missing. The test_hf_registry_coverage test will fail because CwmForCausalLM has not been added to the HF_EXAMPLE_MODELS in tests/models/registry.py. This update is required to ensure CI passes and the new model is properly tracked in tests.
Co-authored-by: Roger Wang <hey@rogerw.io> Signed-off-by: Jacob Kahn <jacobkahn1@gmail.com>
Signed-off-by: Jacob Kahn <jacobkahn1@gmail.com> Co-authored-by: Roger Wang <hey@rogerw.io> Signed-off-by: yewentao256 <zhyanwentao@126.com>
Signed-off-by: Jacob Kahn <jacobkahn1@gmail.com> Co-authored-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Jacob Kahn <jacobkahn1@gmail.com> Co-authored-by: Roger Wang <hey@rogerw.io>
Signed-off-by: Jacob Kahn <jacobkahn1@gmail.com> Co-authored-by: Roger Wang <hey@rogerw.io>
Purpose
Code World Model (CWM) in this collection https://huggingface.co/collections/facebook/cwm-68acbc3eb02570bd89b3aae8 need to be mapped before the Transformers implementation is complete, else we'll have working VLLM mappings that break in HuggingFace Transformers.
HuggingFace Transformers work is WIP -- this will ensure that we can send a CwmForCausalLM config to HuggingFace repos without enabling checkpoint use in Transformers, where it's currently supported.
It also prepares us to eventually use the
CwmForCausalLMclass once released.Test Plan
Local env build + test with HF checkpoints from https://huggingface.co/collections/facebook/cwm-68acbc3eb02570bd89b3aae8 that have had their
config.jsonmodified to useCwmForCausalLMTest Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.